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ABSTRACT 

This model for evaluating the professional activities 
of university professors is based on: (1) a systematic process for 
reporting their activities; and (2) discrepancy evaluation — the 
comparison of goals and objectives with outcomes. The model assesses 
a professor's activity in ten categories — teaching; student 
evaluations; research, creative ability, and scholarship; 
teaching/research; public service; intra-university activity; 
extra-university activity; administration; advisee evaluations; and 
test question evaluations* The evaluation involves 10 basic steps: 
(1) the professor allocates to the 10 categories the percent of 
effort he sets as his goal for the coining semester or year; (2) the 
professor completes the goals and objectives for all categories; (3) 
the department chairiran evaluates the effort allocation and goal 
intent statements; (4) if discrepancies are noted in step 3, the 
chairman and professor eliminate or reduce chem; (5) the Overall 
Evaluation Form is completed by the professor and then by his 
chairman; (6) during the year or semester, the student evaluation 
form is administered to ail the professor's students, the professor 
submits samples of his tests, and evaluation forms are sent to his 
advisees; (7) after the year or semester, the professor completes the 
"description of activities and processes"; (8) the professor then 
completes the "accomplishment" section, the "products and 
accomplishments" section of each category, and a self evaluation; (9) 
the chairman evaluates the discrepancies between goals and 
accomplishments; (10) on subsequent cycles through the system, the 
professor tries to reduce discrepancies, . (For related documents^ see 
TM 003 217-218.) (KM) 
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A Dif^crepancy Evaluation System 
for University Professors 

The evaluation of the professional life of a university professor is 
a difficult task, even for a trained evaluator. The problem is compounded 
for the department head or dean of a college, when this person has no for- 
mal training or background In the field of evaluation. In practice, the 
difficulty in describing and evaluating the professor has led to evalua- 
tion procedures which rely on student evaluations, Intuitjve feelings on 
the part of department chairmen and deans, and the unsystematic recording 
of "professional accomplishments" during the evaluation procedure. This 
work is an attempt to place the assessment procedure on a systematl.: bas- 
is for the reporting, measurement, and evaluation of the activities of 
the university professor. 

This evaluation model rests on a foundation of two basic ideas. First, 
the model attempts to implement a systematic process for reporting the 
activities of university professors. This feature of the model has been 
drawn from the work. Systems for Measur i ng and Report ing the Resources 
and Activities of Colleges ^nd Un t vers i t tes^ publ i shed by the National Science 
Foundation (Henle, 1967). This paper presents a rationale for Identify- 
ing and describing academic and professional personnel, describes the con- 
cept of the "full professional life," and details the activities measure- 
ment data required to describe the university professor. 

Ti*2 second major concept utilized in the development of this evalua- 
tion model Is that of "discrepancy evaluation" (Provus, 1971; Gelsert, 
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1972). Discrepancy evaluation compares the goals and objectives of a sys- 
tem (or professor) with the outcomes of the system. In other words, dis- 
crepancy evaluation compares intents with accomplishments. The two require- 
ments of a discrepdriCy evaluation system are a definitive statement of goals 
and objectives, and some method of reporting and measuring outcomes. In 
tne case of a professor, discrepancy evaluation compares the goals of the 
professor with his accomplishments. This comparison procedure yields 
"discrepancies" between intents and outcomes, and these discrepancies are 
then utilized as data for decision making. In addition, discrepancies be- 
tween the goals of the professor and the goals of the institution can be 
noted and negotiated (for example, a conflict between the professor who has 
strong "teaching goals" and the dean who wishes him to have strong "research 
goa Is"). 

Discrepancy evaluation allows great flexibility In that the professor 
is allowed (in fact encouraged) to set his own professional goals. He is 
then expected to "live up to" his own goals, and providei data that he has 
reached his goals. A discrepancy between goals and outcomes Indicates that 
some change must be instituted In the activities of the professor. A con- 
tinuing discrepancy Indicates that either the goals of the professor are 
not compatible with the needs of the institution, or the professor needs 
systematic help in reaching his goals (for example, a low student rating 
may call for helping the professor to learn some new approaches to instruc- 
tion). 

This evaluation model assesses the activity of the professor in ten 
basic categories. These categories do not exhaust the possibilities for 
evaluation, but they probably do represent the most important areas of 
concern. The evaluation areas are: 
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Teach i ng 

Student Evaluations 

Research, Creative Activity, and Scholarship 

Teach i ng/ Research 

Publ ic Service 

I ntra-Un i vers i ty Activity 

Extra-University Activity 

Adminl stration 

Advisee Evaluations 

Test Queition Evaluations 
The following section provides operational definitions for each of the 
evaluation areas. \ 

Teaching . instruction of students in forma 1 courses and superv I sed 
study. The activity envisioned here is the total activity of teaching, 
not simply the actual contact in the classroom, laboratory, or study. It 
includes directly preparatory activities, such as preparation for class 
and lecture, the setting-up cf demonstrations, and preparation for labora- 
tory experiments. It includes all interaction with students in relation 
to their instruction, such as academic counseling and special review or 
coaching, and all forms of evaluation, such as correcting papers and de- 
termining grades. 

Student eva 1 uat ions . The research is clear that student evaluations 
are a necessary and valuable portion of the evaluation process of univer- 
sity professors. In a survey of all of the research performed on this 
area, the general conclusion was reached: 
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A review of empirical studies indicates that student 
ratings can provide reliable and valid information 
on the quality of courses and instruction. Such in- 
formation can be of use to academic departments in 
constricting normative data for the evaluation of 
teaching and may aid the individual instructor in im- 
proving his teaching effectiveness. 

•..if teaching performance is to be evaluated, either 
for purposes of pay and promotion or for individual 
improvement, a systematic measure of student attitudes, 
op'nions, and observations can hardly be ignored. The 
data which have been reviewed strongly suggest that 
the use of forma 1 s tuden t rat i ngs provi des a reasonab 1 e 
way of measuring student reaction (Costin, F. , Greenough, 
T. , & Menges, R. J., Student ratings of college teach- 
ing: reliability, validity, and usefulness. Review of 
Educat ional Research , Vol. 4l , No. 5, 197 0- 

The evaluation used in this model is the standard University of Wyoming 
Student Evaluation Form. This category could be improved through the uti- 
lization of a form prepared by professional evaluators. 

Resear ch . Methodical study and activity carried on in order to augment 
and develop knowledge rn any field. The definition should be read in its 
broadest possible meaning, and should include the areas of basic research, 
appHed research, and developmental activities. 

Teach i ng/Re search . This term is used to designate that kind of research 
activity which is carried on with one or more apprentice researchers for 
vihom this research involvement Is part of their formal educational program 
and for whom, therefore, the principal investigator plays the role not only 
of "research admi n i strator'* but of preceptor as well. ''Teaching/Research" 
is used, therefore, to distinguish t'le kind of research just defined from 
that research activity in which co-v>forkers and subordinate participants are 
employed not for their own education but simply to assist in carrying out 
the research. It does not include formal instruction about research, i.e., 
courses in research methodology, which would be strictly a teaching function. 



Publ i c Servi ce . This includes direct and indirect service to public 
schools, educational agencies, state department of education, educational 
consultant activities, in-service courses, and services to the community, 
state, and the public of this and other countries. 

Intra-University Activities , All other activities by which one makes 
one's skills and resources available to other persons or organizations with- 
in the university for university purposes (exclusive of activities reported 
in previous categories). 

Extra-University Activities , All activities directed to persons, agen- 
cies, and other organizations outside the university which are not primarily 
public service activities. The main activity here Is consultation for profit- 
making organizations in business and industry. 

Admin i strat ion . The coordination and direction of others in regard to 
the formulation of policy, decision-making, and its implementation. 

A dvi see Eval uat ions , This area is parallel to student evaluations, ex- 
cept it evaluates the activities of the professor in regards to his relation- 
ships to his student advisees. 

Test Question Evaluations , Since test questions are the main source of 
information to the student concerning the goals of a professor for his course 
(if they are not supplied as explicit, written goal and objective statements), 
an evaluation of test questions provides a valuable source of data concern- 
ing the teaching area. 

Performing the Evaluation 
This evaluation involves 10 basic steps. 
Step 1. The professor is to allocate to the various categories of the 
evaluation the percent of effort he would like to set as his 
goal for the coming academic year (or semei^ter) (use the Professional 



Distribution of Effort Form, page 11). This distribution should 
represent a "goal" more than a "c^escr i pt ion" of his effort, since 
the allocations represent his expectations for the future. 
Step 2. The professor completes the statements of "goals and objectives" for 
all categories in the evaluation. He determines his goals for each 
area, being as expliclL as possible. There is some research which 
indicates that the successful self-actualized person is one who 
can set for himself rather carefully stated behavioral goals. 
It Is not easy to state goals, nor is there a good body of 
literature that can "teach" you how to state good goals. There 
are a couple of short books on how to write and analyze goals 
and objectives, but all of these relate directly to one category, 
teaching. These books ara very valuable in this areo, and there 
is a great deal of information in them you can transfer to the 
writing of goals in other areas. The books are: 

Prepar i ng i ns t ruct iona i Object 1 ves , Robert Kager , Fearon Press , 
1962, $1.75. 

Goal Analysis , Robert Mager, Fearon Press, 1972, $2.95. 
Developing Attitudes Toward Learning , Robert Hager, Fearon Press, 
1968, $2.00 

Of course, some categories will have extensive goal statements, and 
some will have no goal statements^ but in total, the statements 
should be a complete description of professional intent. 
Step 3. The department chairman evaluates the effort allocation and goal 
intent statements to determine any discrepancies between the pro- 
fessor's goals and the department's or college's goals. 
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If d i ^^crepanci es are noted in Step 3, negotiations are under- 
tr^ken by the chairman and the professor to eliminate or reduce 
discrepancies. This part of the evaluation parallels the draw- 
ing up of a contract, and the negotiations continue until both 
parties feel the goals represent the best interests of the uni- 
versity and the professor. In this manner, accurate expecta- 
tions can be established in both parties as to the allocation of 
professional effort, and disagreements (i.e,, as to the teaching 
versus research allocation) can be settled prior to final evalua- 
tio,i for promotion, tenure, or raises. 

The Overall Evaluation Form (page 24) is completed first by the 
professor and then by his chairman. 

During the course of the academic year (or semester) the standard 
University Student Evaluation Form wi I ! be administered to all 
the professcir*s students, the professor will be asked to submit 
samples of his tests for analysis by an objective evaluator, and 
evaluation forms will be sent to his advisees. It is assumed 
that the resource of a professional evaluator will be available 
for test evaluation (perhaps through a university instructional 
center). If this is not the case, either this portion of the 
evaluation will have to be bypassed, or a consultant will have 
to be employed to fill this role. These data are compiled and 
placed in the professor's evaluation folder (Student Evaluation 
Form, page 13j Advisee Evaluation Form, page 20, and Test Evalua- 
t ion Form, page 21 ) . 

After a suitable period of professional activity (one academic 
year or semester), the professor completes the "description of 



activities and processes" section of each category. This section 
?s directly parallel to the "goal and objectives" section, and 
the professor should provide data whether or not he has reached 
the goals he set for himself earlier in the year. 
The professor then completes the "accomplishment" section of the 
Professional Distribution of Efforts Form» the "products and 
accomplishments" section of each category^ and his se 1 f-eva luat ion 
of his performance on the Overall Evaluation Form, 
The department chairman evaluates the discrepancies between the 
professor's goal Intent and goal accomplishments. Iri doing this, 
he utilizes the professor's comments under "goals and objectives" 
and evidences he has supplied under "products and accomplishments." 
On the basis of the data supplied by the professor, the chairman 
makes comments and summarizes hfs feelings on the Overall Evalua- 
tion Form. 

If a department chairman wishes to anter adverse comments into 
the evaluation at this time, he Is responsible for documenting and 
providing data which supports his conclusions. Just as the pro- 
fessor Is responsible for demonstrating through the use of data 
that he has reached his goals, the department chairman Is responsible 
for using data to dernonstrate where the professor has failed In his 
professional responslbt 1 i ties. 

The chairman then presents his summary to the professor and 
the total assessment Is discussed. The chairman then submits the 
final evaluation to the dean, who uses all the data to complete 
his section on the Overall Evaluation Form. If the dean wishes 
to enter adverse comments Into the evaluation^ he also Is respon- 
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sible to document and provide data which supports his conclusions. 
Step 10» On subsequent cycles through the system, the professo/ attempts 

to reduce any discrepancies between his goals and the department's 
goals, and between his goal intents and accomplishments. 

This evaluation w* 1 1 also illuminate the goals of the depart- 
ment, and should distinct discrepancies appear between professors' 
t,ioals and department goals, it is clear that in some cases the 
discrepancy should be reduced by a revision of the department's 
goals rather than professors' goals. 
!t should be noted that a discrepancy evaluation procedure is not per- 
formed to label a professor as "good or bad" but rather is used to deter- 
mine when the professor is failing to reach his own goals. In this man- 
ner, evaluation provides the feedback necessary to implement procedures 
to remedy def icien^:! es. For example, should a professor receive low 
student rcitlngs, this evaluation provides feedback that the professor needs 
help with his Instructional procedures. Instead of "letting the professor 
go" for poor student evaluations, a "Center for the Development of University 
Instruction" could be employed to diagnose and remedy teaching deficiencies. 
This concept Is not a new one, and such campus centers are presently upgrau* 
ing instruction at numerous universities. With the feedback provided by 
this evaluation, perhaps university professors will welcome the assistance 
of a center to provide help in the achievement of their professional goals. 
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EVALUATION FORMS 



Pag 

Professional Distribution of Efforts * .... 11 

Teaching 12 

Student Evaluation 13 

Research, Creative Activity, and Scholarship ]h 

Teaching/Research 15 

Publ ic Service ]C 

I ntra-Un i versi ty = .... 17 

Extra-University 18 

Administration 19 

Advisee Evaluation r .20 

Test Evaluation 21 

Overall Evaluation 2k 
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PROFESSIONAL DISTRIBUTION OF EFFORTS 

INTENT ACCOMPLISHMENT 

TEACHING 



RESEARCH, CREATIVE ACTIVITY, 
AND SCHOLARSHIP 



TEACHING/RESEARCH 






PUBLIC SERVICE 






INTRA-UNIVERSITY ACTIVITY 






EXTRA-UNIVERSITY ACTIVITY 






ADMINISTRATION 








100^ 


100% 
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TEACHING 



General Areas Covered 

1. Academic counseling (advisees) 

2. Special preparations, demonstrations, etc. 

3. Preparation of special or innovative programs 
Development of a new course, or revision of an old course 



TEACH ING--GOALS AND OBJECTIVES 



TEACHING-- ACTIVITI ES AND PROCESSES (courses taught, hours spent, 

level of program, type of instruction, number of students, term, 
number of TA's, etc.) 



TEACHING— PRODUCTS AND ACCOMPLISHMENTS 
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STUDENT EVALUATION 



DESCRIPTION OF ACTIVITIES 



Courses evaluated 


Date 


Number of Students 









- - - 

STUDENT EVALUATION GOALS 




GOALS AND OBJECTIVES 


ACCOMPLISH- 
MENTS AND 
PRODUCTS 

AVERAGE 
RESPONSE 


STEM "IDEAL" 
1 AVERAGE 
i RESPONSE 
! THE GOALS AND OBJECTIVES AND ACCOMPLISH- 
I1ENTS "^OR THIS AREA WILL DEPEND ON THE 
EVALUATION FORM WHICH IS USED. 

J 

I 

i 
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RESEARCH, CREATIVE ACTIVITY, AND SCHOLARSHIP 



General Areas Covered 

1. iiescrlbe your research program 

A. Type (applied, pure, developmental) » 

B. Sponsorship (university, federal , state, private, industry, 
individual , other) 

C. Area of study 

2. Published articles (titles, date of publJcation, journal) 

3. Completed books (titles, co-authors, publisher, number of pages, 
date of publication) 

k. Inventions 

5. Works in progress 

6. Membership in learned and professional societies 

A. Names 

B. Offices held (date) 

C. Papers read 

7. Other 



RESEARCH, CREATIVE ACTIVITY, AND SCHOLARSH I P-- GOALS AND OBJECTIVES 



RESEARCH, CREATIVE ACTIVITY, AND SCHOLARSH I P--ACT I V I T I ES AND PROCESSES 



RESEARCH, CREATIVE ACTIVITY, AND SCHOLARSH I P-- PRODUCTS AND ACCOMPLISHMENTS 




TEACHING/RESEARCH 



General Areas Covered 

1. Type of research, number of students involved 

2. Thesis direction 

3. Teaching/research outcomes, articles published, thesis 
completed, etc. 



TEACHING/RESEARCH--GOALS AND OBJECTIVES 



TEACH I NG/REoEARCH— ACT I V I T I ES AND PROCESSES 



TEACH ING/RESEAKCH-- PRODUCTS AND ACCOMPLISHMENTS 
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PUBLIC SERVIC E 



General Areas Covered 

1. Consulting (that which services public interest) 

2. Service on governmental committees 

3. Service on community committees 
^. Editing professional journals 

5. Participating in accrediting procedures 

6. Talks given 

7. Judging contests and competitions 

8. Inter-university relations 

9. Service to public schools 

10. Educat ional agenc ies 

11. State Department of Education 

12. Others 



PUBLIC SERVICE--GOALS AND OBJECTIVES 



PUBLIC SERVICE--ACT1VITIES AND PROCESSES 



PUBLIC SERVICE--PRODUCTS AND ACCOMPLISHMENTS 
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INTRA-UlilVERSITY 



General Areas Covered 



1. Working with student groups in an advisory or supervisory capacity 

2. Working on university boards and committees 

3. Student counseling (other than related to teaching) 
A. University staff activity 

5. University fund-raising 

6. University public relations 

7. Other 



INTRA-UNIVERSITY--GOALS AND OBJECTIVES 



INTRA-UNIVERSITY--ACTIVITIES AND PROCESSES 



INTRA-UNIVERSITY--PRODUCTS AND ACCOMPLISHMENTS 
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E.XTRA-UNIVHRSiTY 



General Areas Covered 

1. Consulting (for prol-'i t-maki ng organizations) 

2. Other 



FYTRA-IINIVERSITY — GOALS AND OBJECTIVES 



EXTRA-UN I VERS I TY-- ACT I V I T I ES AND PROCESSES 



EXT RA- U N I V E RS I TY - - PRODUCTS AND ACCOMPLISHMENTS 
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ADMINISTRATION 



General Areas Covered 

1 . General 

2. Academic 

3. Research 

A. Creative activi ty 

5. Supporting services 

6. Other 



ADM I N I STRAT I ON--GOALS AND OBJECTIVES 



ADM I N I STRAT I ON— ACTI V ITI ES AND PROCESSES 



ADM ! N I STRAT I ON--PRODUCTS AND ACCOMPLISHMENTS 
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ADVISEE EVALUATION 



DESCRIPTION OF ACTIVITIES 



Types of Advisees 


Number of Advisees 







ADVISEE EVALUA T ION GOALS 



GOALS AND OBJECTIVES 



ACCOMPLISH- 
MENTS AND 
PRODUCTS 



ITEM 



"IDEAL" 
AVERAGE 
RESPONSE 



AVERAGE 
RESPONSE 



THE GOALS AND OBJECTIVES 
MENTS FOR THIS AREA WILL 
EVALUATION FORM WHICH IS 



AND ACCOMPLI SH- 
DEPEND ON THE 
USED. 
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TEST EVALUATION 



DESCRIPTION OF TESTS EVALUATED 



COURSES 


TEST SUBMITTED 







TEST EVALUATION GOALS 


GOALS AND OBJECTIVES 


ACCOMPLISHMENTS AND PRODUCTS 
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TEST EVALUATION 



Name of course test is used in. 



Teaching method employed. 



Type of test submitted (weekly quiz, mid-term, 
final , etc. ) 



% of final grade represented by this test. 



Dis /ibution of raw scores on the test. 



Distribution of grades awarded on the test. 



Explain how these questions were chosen. 



Explain how the results of the test were used. 



Explain how the test related to the goals of 
your course. 



Explain how the grade distribution was 
determi ned . 



Remember to attach a copy 
of the test to this form. 
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TEST EVALUATION 



ANALYS I S 

Type of questions: 

Level of quest ions 

Knowledge--recal 1 
Appl i cat ion 
Analysis 
Synthesi s 

Evaluation or Problem Solvinq 

Errors 

In stem of question 

In responses of question 

Ambigu I ty level 

GENERAL IMPRESSIONS 

Grade distributions 

Relationship of test to course goals 

OV ERALL EVALUATION 

^OUTSTAND I NG 

V ERY SATISFACTORY 
^S ATISFACTORY 
IMPROVEMENT NEEDED 



UNSATISFACTORY 
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OVERALL EVALUATION 



AREA SELF 

EVALUAT 


ION 


CHAIRMAN 
EVALUATION 


DEAN 
EVALUATION 




1 

1 ; 2 

i 

L 


3 


h 


5 


f 

1 


2 


3 


k 


5 


1 


2 


3 




5 


TEACHING 
































STUDENT EVAL. 






























RESEARCH 


1 

1 










( 
1 
















TEACHING/RESEARCH 


t 




























PUBLIC SERVIPF 


i 




























\ NTRA-IJM IVFR^ 1 TY 
I n 1 r\r\ uitJ 1 * 1 1 I 
































EXTRA-UNtVERSITY 






























ADMINISTRATION 


1 1 


























ADVISEE EVAL. 


i 


























TEST QUESTION EVAL. 






























1 2 1 3 


h 


5 


1 


2 


:> 


h 


5 




2 


3 


h 


5 

1 



Place an "I" in the appro- 
priate place for an evalua- 
tion of Goal Intent. 

Place an "A" in the appro- 
priate place for an evalua- 
tion of Goal Accomplishment. 

Place an "NA" where item is 
not appl i cable 



1 OUTSTANDING 

2 VERY SATISFACTORY 
:5 SATISFACTORY 

h IMPROVEMENT NEEDED 

5 UNSATISFACTORY 



Department Chairman — General Comments 



Professor -- General Comments 



Dean -- General Comments 
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